Continuous Facial Affect Recognition from Videos

The interpretation of user facial expressions is a very useful method for emotional sensing and constitutes an indispensable part of affective HCI designs. This paper proposes an effective system for continuous facial affect recognition from videos. The system operates in a continuous 2D emotional space, characterized by evaluation and activation factors, enabling a wide range of intermediary affective states to be worked with. It makes use, for each video frame, of a classification method able to output the exact location (2D point coordinates) of the still facial image in that space. It also exploits the Kalman filtering technique to control the 2D point movement along the affective space over time and to improve the robustness of the method by predicting its future locations in cases of temporal facial occlusions or inaccurate tracking. The system has been tuned with an extensive universal database and preliminary evaluation results are very encouraging.